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Background / Context: 

Pretests are a common and useful tool designed to measure what a subject kn ows before 
an intervention or training phase. Tests, however, do not measure knowledge in isolation. The 
act of taking a pretest may change both the subjects’ current knowledge and the way that they 
process the intervention. Many studies have shown that taking a test after learning improves 
subjects’ performance on a later test (see Roediger & Karpicke, 2006 for a review). That is, 
taking a test changes the learner’s current knowledge. The question that we are interested in is 
whether taking a test can also change what the subject leams in the future. 

There is some prior research suggesting that pretests can improve later learning. 

Richland, Kornell and Kao (2009) had subjects study a short scientific text on vision. Before 
reading the text, some subjects answered pretest questions, while others simply read the 
questions and tried to memorize them. On the final test, subjects who had attempted to answer 
the questions on the pretest outperfonned subjects that memorized the questions. This result 
occurred even when looking only at questions answered incorrectly on the pretest. That is, 
answering a question incorrectly on the pretest was more beneficial to later learning than simply 
reading the question. Taking the pretest did not simply measure the students’ prior knowledge, it 
also changed the way that they read and remembered the studied text. 

In other contexts, however, pretests might hinder future learning. If the pretest activates 
not only incorrect information, but also a faulty method for solving the problem or an incorrect 
mental representation, then that incorrect representation may impair later learning. For example, 
in Dunker’s (1945) classic experiments on functional fixedness, subjects were less likely to solve 
a problem by using a matchbox as a platform if they had previously seen the matchbox holding 
matches. Their prior experience with the matchbox (the incorrect mental representation) harmed 
their later problem solving. In the same way, subjects’ experience on a pretest may bias their 
interpretation of an intervention. 

One domain where children may use an incorrect mental representation on a pretest is 
with numerical magnitudes. Young children typically have a logarithmic representation of 
number where small numbers are easily distinguishable, while large numbers are all thought to 
be of a similar magnitude. After further experience with numerical magnitudes children develop 
a linear representation where the difference in magnitude between 3 and 4 is the same as the 
difference between 9 and 10. This shift occurs at different ages for different ranges of numbers; 
during preschool for the numbers 0-10 (Berteletti, Lucangeli, Piazza, Dehaene, & Zorzi, 2010), 
between kindergarten and 2 nd grade for 0-100 (Siegler & Booth, 2004) and between 2 nd and 4 th 
grade for 0 - 1000 (Booth & Siegler, 2006). 

Opfer and Thompson (2008) took advantage of this transition to examine how activating 
an incorrect mental representation might harm later learning. In their experiment, 1 st and 2 nd 
graders were tested on their knowledge of numerical magnitudes for the numbers from 0 to 1000. 
Initially, half of the children received a pretest where they were asked to classify numbers as 
“really small”, “small”, “medium”, “big” or “really big”. Importantly, children in this age range 
typically use the incorrect logarithmic representation to classify numbers between 0 and 1000. 
Then, all of the children participated in a training session where they were asked to place 
numbers on a 0 - 1000 number line and received feedback on their placements. Finally, all of 
the children took a posttest where they again categorized numbers between 0 and 1000 as big or 
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small. 2 Children who had taken the pretest and activated the incorrect mental representation did 
not transfer their learning from the feedback phase and continued to use the logarithmic 
representation on the posttest. Children who received the feedback without taking the pretest, 
however, showed large gains in accuracy and used a linear representation on the posttest. 

Why would incorrect answers be harmful to later learning in the domain of numerical 
magnitudes (Opfer & Thompson, 2008), while they were helpful in learning from a scientific text 
(Richland et ah, 2009)? One large difference is that children in the Opfer study were not only 
answering incorrectly, they were also using an incorrect mental representation to generate those 
incorrect answers. In contrast, the participants in the Richland experiments were generating their 
incorrect answers on the spot and were not using a previously established mental representation 
to guide their answers. We do not believe that it was the incorrect answers on the pretest that 
hindered later learning on the numerical magnitude task; rather it was the children’s continued 
use of an incorrect mental representation. 

Purpose / Objective / Research Question / Focus of Study: 

In this experiment, we sought to further examine the effects of pretests on later learning. 
Specifically, we were interested in if activating a correct mental representation on the pretest 
would improve later learning and if activating an incorrect representation would impair learning. 
Kindergarteners and preschoolers were split into three groups: one group took a pretest designed 
to activate a detrimental logarithmic representation, one took a pretest designed to activate a 
beneficial linear representation, and a control group received no pretest. After a short feedback 
session, all children completed a posttest to determine if they had learned from the feedback. 

Setting: 

The research was conducted at a university preschool and kindergarten in Pittsburgh, PA. 

Population / Participants / Subjects: 

Fourteen preschoolers and eighteen kindergarteners participated in the experiment for a 
total of 32 participants. The children were between four and six years old with an average age of 
5.22 years. The small number group contained 4 preschoolers and 7 kindergarteners (M= 5.43 
years). The large number group consisted of 6 preschoolers and 5 kindergarteners (M= 5.06 
years). The control group consisted of 4 preschoolers and 6 kindergarteners (M= 5.17 years). 

Intervention / Program / Practice: 

There were three phases in the experiment: pretest, feedback and posttest. The pretest 
consisted of 15 trials. On each trial a number was presented above a number line with clearly 
marked endpoints. The child was directed to click the place on the line where that number 
belonged. A green line would appear where they clicked, and then they were instructed to press 
the “Next” button at the top of the screen. This movement to press the button, along with 
instructions from the experimenter, prevented them from clicking in the same spot repeatedly. 
Children who took the small number pretest used a number line that ranged from 0 to 10. 
Children in this age range typically use a linear representation for the numbers 0-10. The large 
number pretest number line ranged from 0 to 20. Typically, children of this age would use a 
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The experiment also contained two control conditions that are not described. 
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logarithmic representation on this number range. Children in the no pretest group skipped the 
pretest and moved directly to the feedback phase. 

During the feedback phase, all children were tested on a 0 to 20 number line. The 
feedback trials consisted of three numbers that were tested on the posttest and three that were not 
for a total of 6 trials. When the children clicked on the number line, a red line indicated where 
they clicked, and a separate line indicated the correct placement. If their guess was less than 
10% away from the correct answer, the line indicating the correct answer was red, and the 
monitor would display the message “Great!”. If their guess was outside that range, then a blue 
line indicated the correct answer and the message displayed was “Not Quite.” The experimenter 
verbally reinforced the feedback and motivated the child to try the next trial. 

On the posttest, all of the children received 15 trials with a 0 to 20 number line. Each 
trial proceeded in the same manner as the pretest and the children received no feedback. For 
children in the large number pretest group the posttest was exactly the same as the pretest. 

Research Design: 

The experiment had a between subjects design with participants receiving either the small 
number pretest, the large number pretest or no pretest. 

Data Collection and Analysis: 

The experiment was conducted in a quiet space at the preschool. The children used a 
laptop with a plugged-in mouse, facilitating easier motor movements than the trackpad. The 
experimenter only assisted the child with computer skills where necessary. All analyses were 
conducted in SPSS and are described below. 

Findings / Results: 

To examine the children’s numerical magnitude representations we relied on two 
measures, accuracy and linearity. 

Accuracy 

The accuracy of children’s estimates was measured using percent absolute error 
(PAE): PAE = (|child’s estimate - correct answer|)/numerical range. For example, if a child 
was asked to place 9 on a 0-10 number line and he placed it at 6, his PAE would be 30% ((|9- 
6|)/10). Note that PAE and accuracy are inversely related. That is, a child with a smaller PAE is 
more accurate at the task. 

(Please insert Figure 1 here) 

As shown in Figure 1, children who received the small number pretest improved greatly 
from the pretest to the posttest, 28% vs. 14%, t(10) = 5.72 , p < .05, d = 1.73. This improvement 
occurred despite the pretest being nominally easier than the posttest, given the smaller range of 
numbers. The children who received the large number pretest, however, showed no 
improvement from the pretest to the posttest, 19% vs. 18%, t < 1. Importantly, children in the 
small number group had smaller errors on the posttest than the control group who did not receive 
a pretest, 14% vs. 23%, t(19) = 1.90 ,p = .07, d = .82 . While the effect is only marginally 
significant, it suggests that including the small number pretest may have increased student 
learning as compared to the feedback phase alone. There was no difference between the large 
number group and the no pretest group on the posttest, t< 1 . 
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Linearity 

We next computed the best fitting linear and logarithmic functions relating the number 
presented to the participants’ estimates. At pretest, the small number group was better fit by the 
linear function than by the logarithmic function, mean R 2 | m = .48 vs. mean R 2 i og = .38, /( 1 0) = 
3.04,/? < .05, d= .92. In contrast, the estimates of the large number group were fit equally well 
by the linear and logarithmic functions, mean R 2 u n = .56 vs. mean R 2 i og = .53, t(10) = 1.29, n.s. 
After the feedback phase, the small number group became more linear in their estimates, while 
there was no change in the linearity of the estimates of the large number group. The percent 
variance explained by the linear function increased from 48% to 68% in the small number group, 
?(10) = 2.15,/? = .06, d = .65, and remained stable at 56% for the large number group, t< 1. 
Again, there was a marginal effect suggesting that the small number group may have learned 
more than the control group. At the posttest, the linear function explained 68% of the variance 
for the small number group compared to 42% for the control group, f(19) = 1.64,/? = .12, d = .71. 
The large number group and the control group did not differ on the posttest, t < 1 . 

(Please insert Figure 2 here) 

Conclusions: 

Activating a correct mental representation before training was beneficial for later 
learning. The small number group, which was significantly linear on the pretest, showed large 
gains following the feedback phase. Their estimates on the posttest were more accurate and were 
better fit by a linear function than their pretest estimates. These improvements came despite the 
posttest being more difficult than the pretest. Most importantly, the small number group was 
more accurate and more linear than the control group on the posttest. That is, activating the 
linear representation on the pretest helped the children to leam more from the feedback phase 
than children who did not receive a pretest. Although these effects are still marginal in 
significance {p = .07,/? = .12), the effect sizes are quite large (d = .82, d= .71) suggesting that 
our small sample sizes are to blame. We are currently planning to run additional participants 
over the summer so the final sample size should be much larger. 

The effect of activating an incorrect mental representation on later learning is less clear. 
The large number group showed no improvement following feedback, however, this lack of 
improvement may have been due to ceiling effects. For unknown reasons, the children who were 
randomly assigned to the large number pretest entered the experiment with greater knowledge of 
numerical magnitudes than the other two groups of children. While we expected the large 
number group to perfonn worse on the pretest than the small number group, they instead 
outperformed their classmates. The three groups did not differ in average age or in the proportion 
of preschoolers versus kindergarteners, so it is unclear why the large number pretest group 
performed so well. Because our manipulation was not successful in activating a logarithmic 
representation in the large number group, we were unable to examine how an incorrect mental 
representation may affect later learning. 

The findings from this experiment, along with the current literature on the positive and 
negative effects of pretests, suggest that researchers need to be careful when designing pretest- 
training-posttest experiments. It is now clear that pretests are not a simple measure of what 
children already know, instead they can alter how children process and remember new 
information. Researchers should be careful to ensure that their conclusions about what children 
did or did not learn during training are not altered by the correctness of the mental 
representations that the children used on the pretest. 
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Appendix B. Tables and Figures 

Figure 1. Mean percent absolute error of children’s estimates on the pretest and posttest. 
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Figure 2. Mean percent variance in individual children’s pretest and posttest estimates accounted 
for by the linear function. 
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